On the Role of Optimization in Double Descent: A Least Squares Study

Empirically it has been observed that the performance of deep neural networks steadily improves as we increase model size, contradicting the classical view on overfitting and generalization. Recently, the double descent phenomena has been proposed to reconcile this observation with theory, suggesting that the test error has a second descent when the model becomes sufficiently overparameterized, as the model size itself acts as an implicit regularizer. In this paper we add to the growing body of work in this space, providing a careful study of learning dynamics as a function of model size for the least squares scenario. We show an excess risk bound for the gradient descent solution of the least squares objective. The bound depends on the smallest non-zero eigenvalue of the covariance matrix of the input features, via a functional form that has the double descent behavior. This gives a new perspective on the double descent curves reported in the literature. Our analysis of the excess risk allows to decouple the effect of optimization and generalization error. In particular, we find that in case of noiseless regression, double descent is explained solely by optimization-related quantities, which was missed in studies focusing on the Moore-Penrose pseudoinverse solution. We believe that our derivation provides an alternative view compared to existing work, shedding some light on a possible cause of this phenomena, at least in the considered least squares setting. We empirically explore if our predictions hold for neural networks, in particular whether the covariance of intermediary hidden activations has a similar behavior as the one predicted by our derivations

Extending the solid step fixed-charge transportation problem to consider two-stage networks and multi-item shipments

This paper develops a new mathematical model for a capacitated solid step fixed-charge transportation problem. The problem is formulated as a two-stage transportation network and considers the option of shipping multiple items from the plants to the distribution centers (DC) and afterwards from DCs to customers. In order to tackle such an NP-hard problem, we propose two meta-heuristic algorithms; namely, Simulated Annealing (SA) and Imperialist Competitive Algorithm (ICA). Contrary to the previous studies, new neighborhood strategies maintaining the feasibility of the problem are developed. Additionally, the Taguchi method is used to tune the parameters of the algorithms. In order to validate and evaluate the performances of the model and algorithms, the results of the proposed SA and ICA are compared. The computational results show that the proposed algorithms provide relatively good solutions in a reasonable amount of time. Furthermore, the related comparison reveals that the ICA generates superior solutions compared to the ones obtained by the SA algorithm

Brain morphology predicts social intelligence in wild cleaner fish

It is generally agreed that variation in social and/or environmental complexity yields variation in selective pressures on brain anatomy, where more complex brains should yield increased intelligence. While these insights are based on many evolutionary studies, it remains unclear how ecology impacts brain plasticity and subsequently cognitive performance within a species. Here, we show that in wild cleaner fish (Labroides dimidiatus), forebrain size of high-performing individuals tested in an ephemeral reward task covaried positively with cleaner density, while cerebellum size covaried negatively with cleaner density. This unexpected relationship may be explained if we consider that performance in this task reflects the decision rules that individuals use in nature rather than learning abilities: cleaners with relatively larger forebrains used decision-rules that appeared to be locally optimal. Thus, social competence seems to be a suitable proxy of intelligence to understand individual differences under natural conditions.info:eu-repo/semantics/publishedVersio

Shrimp closed-loop supply chain network design

Recent developments in food industries have attracted both academic and industrial practitioners. Shrimp as a well-known, rich, and sought-after seafood, is generally obtained from either marine environments or aquaculture. Central prominence of Shrimp Supply Chain (SSC) is brought about by numerous factors such as high demand, market price, and diverse fisheries or aquaculture locations. In this respect, this paper considers SSC as a set of distribution centers, wholesalers, shrimp processing factories, markets, shrimp waste powder factory, and shrimp waste powder market. Subsequently, a mathematical model is proposed for the SSC, whose aim is to minimize the total cost through the supply chain. The SSC model is NP-hard and is not able to solve large-size problems. Therefore, three well-known metaheuristics accompanied by two hybrid ones are exerted. Moreover, a real-world application with 15 test problems are established to validate the model. Finally, the results confirm that the SSC model and the solution methods are effective and useful to achieve cost savings

Reinforcement Learning-Based Dynamic Power Management of a Battery-Powered System Supplying Multiple Active Modes

Abstract-This paper addresses the problem of extending battery service lifetime in a portable electronic system while maintaining an acceptable performance degradation level. The proposed dynamic power management (DPM) framework is based on model-free reinforcement learning (RL) technique. In this DPM framework, the Power Manager (PM) adapts the system operating mode to the actual battery state of charge. It uses RL technique to accurately define the optimal battery voltage threshold value and use it to specify the system active mode. In addition, the PM automatically adjusts the power management policy by learning the optimal timeout value. Moreover, the SoC and latency tradeoffs can be precisely controlled based on a userdefined parameter. Experiments show that the proposed method outperforms existing methods by 35% in terms of saving battery service lifetime. Keywords-Dynamic power management; reinforcement learning, extending battery lifetime; battery-powered system design

Progress towards an accurate determination of the Boltzmann constant by Doppler spectroscopy

In this paper, we present significant progress performed on an experiment dedicated to the determination of the Boltzmann constant, k, by accurately measuring the Doppler absorption profile of a line in a gas of ammonia at thermal equilibrium. This optical method based on the first principles of statistical mechanics is an alternative to the acoustical method which has led to the unique determination of k published by the CODATA with a relative accuracy of 1.7 ppm. We report on the first measurement of the Boltzmann constant by laser spectroscopy with a statistical uncertainty below 10 ppm, more specifically 6.4 ppm. This progress results from improvements in the detection method and in the statistical treatment of the data. In addition, we have recorded the hyperfine structure of the probed saQ(6,3) rovibrational line of ammonia by saturation spectroscopy and thus determine very precisely the induced 4.36 (2) ppm broadening of the absorption linewidth. We also show that, in our well chosen experimental conditions, saturation effects have a negligible impact on the linewidth. Finally, we draw the route to future developments for an absolute determination of with an accuracy of a few ppm.Comment: 22 pages, 11 figure

New crystal packing arrangements in radical cation salts of BEDT-TTF with [Cr(NCS)6]3− and [Cr(NCS)5(NH3)]2−

BEDT-TTF forms three packing arrangement styles in its radical cation salts with [Cr(NCS)6]3− in two of which two trans-oriented isothiocyanate ligands penetrate the BEDT-TTF layers either at the point where a solvent (nitrobenzene) is incorporated in a stack of donors or by four donor molecules forming a “tube” motif to accept a ligand at each end along with a small solvent molecule in between (acetonitrile). The [Cr(NCS)5NH3]2− ion forms a related crystal packing arrangement with BEDT-TTF with a reduction in the number of “tube” motifs needed to accept an isothiocyanate ligand